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This Positive Behavior Interventions and Supports (PBIS) school status report provides a summary of the current status, 
of all schools implementing PBIS in Minnesota, including participation in the Minnesota Department of Education’s 
training sequence, implementation fidelity assessment scores, and outcomes (as appropriate). In particular, this report 
highlights the patterns and trends in PBIS implementation and outcomes in Minnesota, and helps to understand the 
relationships between training, implementation fidelity, and outcomes. This report is intended for all stakeholders to 


better understand the status of PBIS in Minnesota. 
About PBIS Schools 
Currently, there are 449 active PBIS schools in 


Minnesota. The charts below illustrate number of 
schools by region, grade level, and training cohort. 


Number of schools (by region) 
Region Number Percent 
Metro 2m 47% 
North 136 30% 
South 401 2% 
Total 448 100% 
Number of schools (by cohort) 

Cohort Number Percent 
4 (2005) 9 2% 

2 (2006) 9 2% 

3 (2007) 30 T 
4(2008) 2 T 

5 (2009) 44 10% 

6 (2010) B 16% 
72011) 68 15% 
8 (2012) ™ 16% 

9 (2013) ST 13% 
10 (2014) 55° 12% 
Total 448 100% 


* Note that Goodhue County Education District in the South Region 
hhas 3 programs, but scores are reported as one school this year. 
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Number of schools (by grade level) 

Grades served Number Percent 
Early childhood 4 1% 
Elementary 253 53% 
Middle school eT 14% 
High school 60 13% 
Early childhood/elementary 1 = 
Early childhood-7th 1 = 
Early chldhood-8th 1 = 
kB 9 2% 
Kaz 19 4% 
Middle schoolhigh school 12 3% 
ALC 18 4% 
Other" 3 = 
Inactive 2B 8% 
Closed" 2 = 
Total" 478 100% 


*Other categories include schools that serve grades 7-12 and 5-8. 
“The total number includes inactive and closed schools. 


Training participation (data for schools 
currently in the 2-year sequence) 


For schools that are currently participating in the 2-year 
PBIS training sequence sponsored by the Minnesota 
Department of Education (Cohort 9 and 10 schools), 
three indicators are used to illustrate how fully schools 
are participating and invested in the training: team 
attendance at training, administrator participation at 
training, and usefulness of training. 


Team attendance and administrator participation at training 


Schools in the first year of training in 2014-15 (Cohort 10) 
had six full days of training that they were expected to 
attend, Schools in the second year of training in 2014-15 
(Cohort 9) had six days of training in their first year and 
three full days of training in the second year for a total 
of nine days throughout their 2-year training sequence. 


Support from administration is a key feature of schools 
that successfully implement PBIS. Schools that are 
participating in the 2-year training sequence are expected 
to bring an administrator (principal, viee-prineipal, or 
assistant principal) to training. 


Schools and administrators in both cohorts had high rates 
of training attendance. Administrators in the metro 
region in Cohort 9 had the lowest attendance rate, with 
an average administrator attending 6 full days of 
training. Similarly, the north region in Cohort 10 had 
lower administrator attendance when compared to schools 
in other cohorts and regions, while team average 
attendance days are similar across regions. 


Average number of training days by team and administrator 

(Cohort 9 - Year 2 in training sequence, max. = 9.0 days) 
9.0 a7 9$089 89 
75 72 
6.0 
Metro North South State 

(N=27)  (N=14)—(N=16)_— average 
(N=57) 


=Team = Administrator 


Average number of 
(Cohort 10 - Year 1 in training sequence, max. = 6.0 days) 


60 58 6060 59 
56 ; 
46 a 
Metro North South State 
(N=21)  (N=19)— (N=13) average 
(N=53) 
=Team = Administrator 


Usefulness of training 


School teams were asked to provide feedback on the 
training sessions they attended. Participants were asked 
how much they agree or disagree with the following 
statement: “Overall, this training was a positive, worthwhile 
learning experience.” On average, school teams rated the 
trainings rather highly. Schoo! teams from the north and 
south regions in Cohort 10 were more likely to disagree 
that the training was a positive, worthwhile learning 
experience. The graph below illustrates average ratings 
from the fall, winter, and spring trainings combined for the 
2014-2015 school year. 


“Overall, this training was a positive, worthwhile learning 
experience.” (Combined results, all trainings in 2014-15). 


7% 


a 
He 


Metro | North | South | Metro | North 
Cohort 9 Cohort 10 


=Disagree = Agree. 


= Strongly agree 


Implementation fidelity measures 


PBIS is an evidence-based approach to addressing 
behavior issues in schools. A significant amount of 
research has been done to identify the critical features of 
PBIS. More generally, implementation science points to 
a specific sequence to ensure the model is implemented 
with fidelity. The following assessment tools are 
currently used to assess PBIS implementation fidelity 
among participating schools in Minnesota: SET scores, 
BoQ scores, TICs, and SAS. Next school year, the 
Tiered Fidelity Inventory (TF1) will be used for Cohort 
11 and will replace the SET, TIC, and SAS as a way to 
measure implementation fidelity across all tiers of 
PBIS. 


SET scores 


The School-wide Evaluation Tool, or SET, is used to 
measure implementation fidelity of tier | (universal tier) 
of PBIS, At the beginning of their training sequence, 
schools are asked to complete a baseline SET, which is 
administered by a trained SET Evaluator who is from 
outside the school. This baseline score is an indication 
of how much work the school has to do to get from 
where they are currently at to full implementation of 
PBIS. A school is considered to be implementing PBIS 
with fidelity if they score an average of 80 or higher on 
their post-training SET. 


Average SET scores for Cohorts 1-8 (Schools out of training) 


Schools that are out of training should complete a SET 
or a BoQ at least once a year to measure 
implementation fidelity. Seventeen schools from 
Cohorts 1-8 completed a post-training SET evaluation 
in the Spring of 2015. Those schools had an overall 
average score of 91, which indicates that these schools, 
are implementing PBIS with fidelity. 


For schools in Cohort 9, the average SET scores 
increased each time a SET was conducted, from the 
time of their first baseline SET to their final SET. 
Cohort 10 has had two SETs to date, and the average 
SET score of the mid-training SET is higher than the 
average baseline SET score. 


Average SET scores for Cohort 9 
(Schools in year 2 of training sequence) 


a1 
70 80 
Baseline Mid-training- Post - Year 2 
(Fall 2013) Year 1 (Spring (Spring 2015) 
(N=49) 2014) (N=51)—— (N=52) 
Average SET scores for Cohort 10 
(Schools in year 1 of training sequence) 
84 
70 
Baseline Mid-training-Year 1 


(Fall 2014) (N=49) (Spring 2015 N=51) 


Number of Team Implementation Checklists completed 


The Team Implementation Checklist, or TIC, is a tool 
used by schools to measure PBIS implementation fidelity. 
The TIC is supposed to be completed by the PBIS. 
teams in training at least three times per year to monitor 
activities for implementation of PBIS in a school. 
Teams out of training should complete a TIC yearly. 
Only TIC scores from the most recent school year are 
included. 


This past school year, the majority of school teams in 
Cohort 9 (75%) and Cohort 10 (83%) completed three 
or more TICs. However, the majority of schools in 
Cohorts 1-8 (80%) did not complete any TICs this 
school year. See below for the number of schools 
completing TICs in the past school year. 


Number of TICs completed during last school year complete a SET every third year, However, this year 
(by cohort) the eligibility criteria changed so that all schools out of 

1% training (Cohorts 1-8, N=336) were eligible to take the 
% BoQ. Only BoQ scores from the 2014-2015 school year 
are included in this report. Of those that completed the 
BoQ in the 2014-2015 school year, the majority of 
schools (83%) are implementing PBIS with fidelity (as 
indicated by a score of 70 or higher). However, over 
half (60%) of schools that qualified did not complete 
the BoQ assessment (n=202). 


Cohorts 1-8 Cohort9 Cohort 10 All cohorts 


(N=336)  (N=57) —(N=53)—(N=446) BoQ score of schools in Cohorts 1-8 that completed the BoQ 
sOTICs s1TIC =2TICs =3+TICS (2014-2015 school year) (n=134) 
34% 27% 
Number of Self-Assessment Surveys completed 22% 
—_ = - a 
The Self-Assessment Survey, or SAS, examines the status —l™ aa 


and need for improvement of four behavior support systems: >60 points 60-69 70-79 80-89 90+ points 
school-wide, non-classroom, classroom, and individual (N=10) points ee pee (N=36) 
students. A summary of the SAS results should be used to (Nata): (Nest) (Nea) 

develop an action plan for implementing and sustaining 
PBIS systems throughout the school. The SAS should be 
completed by teams in training twice during their first 
year and once during their second year of training. 
Schools out of training should complete the SAS on a 
yearly basis. 


Behavioral data system 


The presence of a behavioral data system allows for 
data-based decision-making at the school PBIS team 
level. The data collected in these systems are used for 
improving school-wide behavior support. Most schools 
Number of SAS completed during last school year in Minnesota that are doing PBIS use the School-Wide 
(by cohort) Information System (SWIS), but a significant 

1% proportion uses other systems. 


Behavioral data system used (Cohorts 1-8, 9 and 10) 


4% 
68% - 10% 5% 2% 
a &.. 


Cohorts 1-8 Cohort9 Cohort 10 All cohorts 
(N=336)  (N=57) — (N=53)—(N=446) 


=0SAS =1SAS «2+ SAS 


Cohorts 1-8 Cohort9 Cohort 10 


Benchmarks of Quality 
= SWIS and another system (N=34, 3, 1) 

Benchmarks of Quality, or BoQ, is a self-assessment a Nannie (N16: 38, 418) 

tool that schools use to assess implementation fidelity eens iaacdsse) 

of tier | of PBIS. Previously, schools that have reached dee 

a score of 80+ on the SET and have completed the 2- System unknown (N=42,2, 41) 


year training sequence are eligible to complete the BOQ ote: on Sis stems include TES, inte Canpus, CLASSROOM, 
every year for two years and then they only need to SKYWARD, and custom systems. 


Outcomes — Triangle data 


One goal of PBIS is to increase the efficiency with 
which schools respond to students” behavioral support 
needs (Sailor, Dunlap, Sugai, & Horner, 2009). This 
efficiency can be maximized by minimizing the number 
of students who need support beyond the universal level. 


The smaller the percentage of students who require 
support beyond the universal level, the more efficiently 
those students can be served with the limited resources 
schools have available. “Socially successful” is defined 
as having 0-1 office discipline referrals (ODR) within a 
given time span, while “needing targeted interventions” 
is defined as having 2-5 ODR, and “needing intensive 
interventions” as having more than 6 ODR within a given 
time span (Homer, Sugai, Todd, & Lewis-Palmer, 2005). 


Data are reported for schools in cohorts 1-10 that have 


entered data through two methods: either through SWIS, or if 
a school uses another type of data system, they were asked to 


Triangle data (Statewide, N=190) 


6 or more ODRs 


—1 4% —1 4% 14% 


Issues to consider 


Triangle data (Metro, N=83) 


© 2to5ODRs 


enter their Office Discipline Referral (ODR) data in an online 
survey maintained by Wilder Research, Results were 
combined and averaged for each region and statewide. The 
triangle scores are calculated using “Majors” only. Major 
behaviors are defined as disciplinary incidents that must be 
handled by administration. These may include, but are not 
limited to: physical fights, property damage, drugs, 
weapons, tobacco, ete. 


‘The hallmark conceptual triangle of PBIS suggests that, 
when PBIS is implemented effectively, on average, 80- 
90% of students are socially successful with universal 
support alone, S-10% require additional support in the form 
of targeted interventions, and 1-5% require intensive 
individualized support (Sugai, Horner, & Gresham 2002). 
Based on this, Minnesota schools that have been trained. 
to implement PBIS are, on average, within the targets. 


Triangle data (North, N=69) Triangle data (South, N=38) 


™ 0to10DR 
2% 
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School participation. Many schools (n=128) that are out of training in Cohorts 1-8 did not submit either 


implementation fidelity assessment scores or discipline data. The SLT and Wilder Research should take a look at 
how long these schools have not been submitting any data and decide if they should be included in future analysis 
for reporting. RIPs could also consider inviting these schools to get reconnected with PBIS and offer incentives 
such as coaching, sustainability training, free TFI or BoQ facilitation, support with data systems, etc., so these 
schools will consider re-engagement in PBIS. Research has shown that using data about their implementation of 


PBIS as well as bout their outcomes (behavior data) is the most critical factor in determining PBIS sustainability at 
the school level.’ 


Training attendance. Administrator attendance at training by Cohort 9 schools dropped when schools entered their 
second year of training, The SLT and trainers should continue to emphasize the importance of administrator 
involvement and support, particularly as schools enter their second year. The SLT could also investigate why 
administrators feel less compelled to participate in training in later stages. This could be done informally at 
trainings in a conversational setting with teams or perhaps RIPs or trainers could reach out to schools to discuss 
with them individually. Another option would be to ask a question in the training survey to gauge administrator 
involvement. 


Data collection and Benchmarks of Quality (BoQ). This year, the SLT changed the criteria that schools needed 
to have in place in order to take the BoQ. Previously, schools had to have completed the two-year training sequence 
and score at least an 80 on their most recent SET. The SLT changed this criteria so schools only have to be out of 
training to take the BoQ—the SET score requirement is no longer in place—in hopes that more schools would take 
the BoQ and report data. While many schools (n=134) completed a BoQ assessment, 202 did not complete this 
assessment, It is important to note, however, that 17 schools out of training did complete a SET. Additionally, a few 
(n=5) schools completed both the SET and the BoQ. Wilder Research will work with the SLT to come up with 
ideas on how to promote the BoQ (such as emphasizing that it is a self-assessment) and will consider updating the 
training material, to hopefully encourage more eligible schools to complete a BoQ. 


‘The Tiered Fidelity Inventory. As the SLT rolls out the TFI as a new way to collect data for cohorts in training, 
RIPs and trainers should find ways to involve schools in all cohorts in this new implementation fidelity tool at data 
days, coaches meetings, on the PBIS website, and otherwise make it known that this is an available tool to use. The 
TEI workgroup has plans to create a webinar to train schools to conduct the TFI. This could appeal to schools out of 
training as it replaces three other implementation fidelity measures (TIC, SAS, and SET). Furthermore, schools in 
training could benefit from the knowledge and expertise of PBIS coaches from older cohorts, so we recommend 
incorporating matching between experienced PBIS schools and newer PBIS schools through a TFI no-cost 
exchange process for the site visits that are intended to take place as a part of the follow-up TFls starting in Spring 
2016. Wilder Research could assist in facilitating this process similar to how we have facilitated the no-cost 
exchange for SET assessments in previous years. . 


Melntosh, K., Predy, L., Uprety, G., Hume, A., Turri, M., and Mathews, 8. (2013). Perceptions of Contextual Features Related to 
Implementation and Sustainability of School-wide Positive Behavior Support. Journal of Positive Interventions, 16(1), 31-43, 
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